1. Field
One or more embodiments relate to a rate-distortion optimizing method that is used to encode a depth image to generate a stereoscopic image.
2. Description of the Related Art
Recently, demand for three-dimensional (3D) images that allow users to view TV, movies, and the like, in 3D space has been rapidly increasing.
A stereoscopic image is a three-dimensional (3D) image that simultaneously provides information associated with depth and space. Stereoscopic schemes may provide stereoscopic effects with two or more images, for example, using a stereo image that merely provides a first image to one eye and a second image to a second eye, or using several images at different points of view to the left eye and right eye. With the latter approach, the several images may provide the stereoscope effect by being projected in different directions according to the point of view of an original camera or a point of view of the viewer relative to the centerline point of view. Here, the viewer observes different images as the viewer moves across the viewing area of the display. In such an arrangement with multiple images, a multi-view image may be used, where there are plural displayed images for potentially each pixel. The viewer would only observe the appropriate left and right images based on the current position of the viewer because of the display surface directs different stereo images in predetermined different directions. This approach of the stereoscopic schemes is called an autostereoscopic approach, with examples including the use of a parallax-barrier or lenticular lenses, to control the direction images can be observed from. With this arrangement, the multi-view image requires substantially more data to be available to generate the different points of view than needed in the stereo image apparoach.
Accordingly, corresponding multi-view images photographed at various points of view, such as by plural cameras arranged at different viewpoints, may be used to provide an image providing different looks as a viewer moves and changes its point of view. However, with the above noted two image stereoscopic approach, the point of view (or viewpoint) would be fixed, while with the autostereoscopic approach several points of view may be observed. With autostereoscopic approach systems, there is a limit in the number of points of view that may be observed by the viewer, which corresponds to potential mechanical limitations of the displays, bandwidth for the additional images, and processing capability for rendering all of the images, and potentially interpolating between different images representing neighboring points of view. Additionally, applications using the images photographed at various points of view may further include a free-viewpoint TV, a 3D TV, and the like. The 3D TV may change images projected to a viewer to enable the viewer to recognize a realistic 3D depth, such as in the above stereo image and the multi-view image projected according to the autostereoscopic approach.
The free-viewpoint TV (FTV) may obtain and analyze a multi-view image with respect to the same scene, and may change the observed point of view for an object. With the above noted autostereoscopic approach, the number of points of view (or viewpoints) and corresponding images that would be transmitted would at least be limited by the capability of the display to differently project each different image. Rather, with the free-viewpoint TV, the number of views that may be selected from for display could be unlimited as some implementations may permit the viewer to control the point of view, thereby permitting the selection of points of view, which could include selecting a point of view behind, above, below, or any other available point of view for an observed object. The number of points of view that may need to be available to such a system could be substantially greater than the autostereoscopic approach system, with the limitations in the number of points of view for the free-viewpoint TV being primarily caused by the underlying infrastructure. However, there are typically substantial difficulties in securing the necessary network infrastructure, terrestrial bandwidth, and the like to compress, transmit, and receive such a massive amount of data for the increased number of images photographed at the various points of view, which affects both 3D TVs that utilize an autostereoscopic approach and such free-viewpoint TVs.